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ABSTRACT 

In this paper we analyze the maximum entropy image deconvolution. We show that given the 
Lagrange multipher a closed form can be obtained for the image parameters. Using this solution 
we are able to provide better understanding of some of the known behavior of the maximum 
entropy algorithm. The solution also yields a very efhcient implementation of the maximum 
entropy deconvolution technique used in the AIPS package. It requires the computation of a 
single dirty image and inversion of an elementary function per pixel. 

Subject headings: Radio interferometry, image formation, maximum entropy, deconvolution 



1. Introduction 

Radio astronomical imaging using earth rota- 
tion synthesis radio telescopes is an ill-posed prob- 
lem due to the irregular sub-Nyquist sampling of 
the Fourier domain. During the last 40 years 
many deconvolution techniques have been devel- 
oped to solve this problem. These algorithms are 
based on models for the radio image. Among 
th ese techn i ques we find the CLEAN method 
by iHogbomI ( 1974). The maximum entropy algo- 
rithm (MEM) bvlFriedenI (11972'). 'G ull and Daniel] 
(ll978l) . lAby (Il974l ) and lWernecker(fr977l) . exten- 
sions of the CL EAN to support multi-resolutio n 
and wavelets b y Bhat nager and Cornwell (2004'), 
Cornwell (,20081 ) and fCornwell et al. ( 2008) . non- 
negative least squares bv Briggs ( 19951). paramet- 
ric ba sed imaging by Leshem and van der Veen 
(|2000f) and ben-David and Leshem (2008,) an d 



sparse Li r econstruction bv iLevanda and Leshem 
(|2008l ) and lWiaux et al.l (|Submitted 20091 ). While 
there is a major amount of experience in using 
these algorithms, we still lack a comprehensive 
theoretical analysis. This would become a more 
critical problem for the future generation of radio 



interferometers that will be built in the next two 
decade such as the square kilometer array (SKA), 
the Low Frequency Array (LOFAR), The Allen 
Telescope Array (ATA) , the Long Wavelength Ar- 
ray (LWA) and the Atacama Large Millimeter 
Array (ALMA). These radio-telescopes will in- 
clude many more stations, will have significantly 
increased sensitivity and some of them will oper- 
ate at much lower frequencies than previous radio 
telescopes, and therefore would be more sensitive 
to modeling and calibration errors. 

The maximum entropy image formation tech- 
nique is one of the two most popular deconvolu- 
tion techniques. The ma ximum entrop y princi- 



tion tecnniques. ine ma ximum entrop y princi- 
ple wa s first proposed bv Ijavne"^ ( 19571 ). Javned 
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(|l982n provides a good overview of the philosophy 
behind the idea. Since then it has been used in 
a wide spectrum of imaging problems. The basic 
idea behind MEM is the following: Among all im- 
ages which are consistent with the measured data 
and the noise distribution not all satisfy the posi- 
tivity demand, i.e., the sky brightness is a positive 
function. Consider only those that satisfy the pos- 
itivity demand. From these select the one that is 
most likely to have been created rando mly. T his 
idea has also been proposed bv iFriedenl ( 19721 ) for 
optical ima ges and applied to radio astronomical 
imaging in ICull and Danielll (|l978l ). Other ap- 
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proaches based on the dif f erenti al ent ropy have 
also b een proposed lAblej (|l974l ) and IWerneckd 
(jl977l ). An extensive collection of papers dis- 
cussing the various methods and aspects of maxi- 
mum entropy c a n be found in the various papers in 
Robertd (|l984l ). iNaravan and Nitvanandal (|l986l ) 
provides an overview of various maximum entropy 
techniques and the use of the various options for 
choosing the entropy measure. Interestingly, in 
that paper, a closed form solution is given for the 
noiseless case, but not for the general case. 

The approach of Gull and DanieiH ( 1978h begins 
with a prior image and iterates between maximiz- 
ing the entropy function and updating the fit 
to the data. The computation of the image based 
on a prior image is done analytically, but at each 
step the model visibilities arc updated, through 
a two-dimensional Fourier transform. This type 
of algorithm is known as a fixed point algorithm, 
since it is based on iterating a function until it 
converges to a fixed point. While it is known 
that for the maximum entropy, this approach usu- 
ally converges, it was re cognized that the conver- 
gence can be slow (see Naravan and Nitvanandal 
Il986l ). Hence, improved methods based on New- 
ton method and the Conju gate Gradient tech- 
nique have Ijeen proposed by (ICornwell and Evand 
198,4 ISaultlll99(ilSkilling and Brvanlll984 . These 
methods perform direct optimization of the en- 
tropy function subject to the constraint. 

In this paper we will provide a closed form 
solution for the maximum entropy image forma- 
tion problem. This solution provides a novel short 
proof of the uniqueness of the maximum entropy 
solution and allows us to provide a theoretical ex- 
planation to the failure of the maximum entropy 
algorithm in cases of strong point sources. The 
explicit expressions for the solution allows us to 
quantify the effect of the free parameters involved 
in the maximum entropy algorithm. Using the 
closed form solution we will also develop a new 
technique for solving the maximum entropy de- 
convolution with a fixed number of operations per 
pixel, except an initial step of gridding, deconvo- 
lution and c omputation o f a si ngle d irty image as 
described in iTavlor et al.l (see 19991 ). We believe 
that this paper is a significant step forward in un- 
derstanding image deconvolution techniques. 



2. The Maximum Entropy algorithm 

We begin with a short description of the max- 
imum entropy algorithm. Following the standard 
convention in this field ( Gull and Daniell 119781: 
Cornwell and Evani 1985 : Saultl 1990 ) we present 
the one dimensional case. Similar results for the 
two dimensional case are also valid (but require 
more complicated notation). 

Gull and Danielll ( 1978f ) showed that the maxi- 



mum entropy solution is given by solving the fol- 
lowing Lagrangian optimization problem: 



Y^Ulogh-^X^V), (1) 
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where 



model 



(2) 



and A is a Lagrange multiplier for the constraint 
that the model based visibilities should match the 
measured visibilities. Taking the derivative with 
respect to we obtain that the solution is given 
by: 



h = 



exp<^ -1 + AE 



model 



'(3) 

where the complex exponent comes from the 
Fourier transform relationship between y^niodel 
and the intensity at direction i. The basic maxi- 
mum entropy algorithm now proceeds by choosing 
an initial image model (typically flat image) com- 
puting the model based visibilities y^^iodel-j 
the grid A. Using these visibilities a new model 
image is computed by equation ([3]). New visibil- 
ities are computed from the new model and the 
process is iterated until convergence. 

3. Closed form solution of the Maximum 
Entropy equations 

We will now show that there is a much sim- 
pler technique to compute the maximum entropy 
image than the existing techniques. The discus- 
sion will also provide a novel simple proof for the 
uniqueness of the maximum entropy solution as 
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well as a very efficient solution. The solution de- 
pends on the Fourier inversion nature of the inter- 
ferometric measurement equation, together with 
the common practice of gridding and interpolating 
the data to obtain simple fast Fourier transform 
between visibility domain and image domain. This 
appro ximation was used by fCornwell and Evan^ 
( 19851 ) where the approximation was computed 
in the image domain using the gridded data. This 
will enable us to show that the unique maximum 
entropy function can be computed independently 
for each pixel, given a first step of data interpo- 
lation in the visibility domain. To that end as- 
sume that I — {Ii : £ G L) is a fixed point of the 
maximum entropy iteration ([3]). Therefore, the 

visibilities Vj™°'^^'^ are just Fourier transform of 
I sampled at the points k G A, where A is the 
rectangular grid. To simplify the discussion we 
will assume that the measurement noise variance 
al = CT^ is identical for all visibilities (an assump- 
tion that follows from standard noise calibration 
practice in radio interferometry) and that a uni- 
form weighting is used, (we will comment in the 
end on other weighting schemes). The right hand 
side of ([3]) is just a Discrete Fourier Transform 
of the difference between the measured visibilities 
and the modeled visibilities. Hence the right hand 
side is actually the difference between the dirty 
image if based on the gridded visibilities and the 
model image at the point Ig . Hence the fixed point 
of the maximum entropy procedure must satisfy 
for each pixel If. 



A 



Ie = cxp\-l + -^N{lf -le] 



(4) 



Alternatively this expression can be derived using 
Parseval identity, and deriving the term with 
respect to le in the image domain. Further sim- 
plification yields: 
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NX 



logh+'-^It^'-^It" -1 (5) 
Taking exponent of the two sides yields 

(6) 
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The function ,fa{x) ~ xe""^ is monotonically in- 
creasing for a > and all x, non negative for all 
positive values of x and negative for negative val- 
ues of X. Since the right hand side of ([6|) is always 



positive every pixel in the reconstructed image is 
positive. By monotonicity of faix) we obtain that 
/ is invertible and the fixed point is uniquely deter- 
mined given the Lagrange multiplier. The solution 
is now given by: 
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/, = f-' I exp { '-^If - 1 
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where a = NX/a'^. Equation (O is very satisfy- 
ing. It provides us with the desired understanding 
of the way the maximum entropy solution manip- 
ulates the gridded data. For each pixel of the dirty 
image if we obtain that if 1 << NXlf jo'^ then 



Nxn 
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Hence whenever the dirty image is sufficiently 
strong we give preference to the dirty image. If 
on the other hand, the dirty image is weak, then 
there is higher risk of obtaining a sidelobe, so the 
prior is more significant, and sidelobes are more 
significantly suppressed. Note that if we have a 
very strong point source in the field of view, its 
sidelobes are also likely to be stronger than other 
emission in the image, and therefore the algorithm 
will not be able to suppress these sidelobes. This 
provides full explanation for the problems of maxi- 
mum entr opy deconvolution a s described by Briggs 
et al. (see Taylor et al. 19991 chapter 8). 



4. Extensions 

The constraint is in many cases insufficient, 
and either a reference image or a constraint on 
the total flux is added in order to provide better 
fit is added. Our solution naturally extends to 
these cases. We will demonstrate it in the first 
case. Assume that we have a reference image. The 
problem ([1]) can be reformulated as 



/ = arg max/ - le log 



IE 



Vk - v. 



model 



(9) 

where Fi is a reference image (possibly low reso- 
lution image we already have). The ratio is taken 
from the expression for the relative entropy or the 
KuUback-Leibler divergence. Taking the deriva- 
tive with respect to Ig and assuming again that 
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ai — (7^ results in the following solution 
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h = cxp |-1 + + [1^ - I, 
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where a = ja^ and ^(x) is computed as 
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Table 1: Exact maximum entropy algorithm 
Initialization: 

Choose a prior image F^. 
Choose A 

Grid the data to obtain Vfe : /e e A 
where A is a uniform rectangular grid. 

Compute the dirty image I^ . 

Image deconvolution: 

For each pixel i 

Compute the closed form solution based on ([TT|) . 
When ip is negative: 

choose the solution with maximum entropy. 

End. 

Compute fit to data. 

If fit is poor, increase A and redo deconvolution. 
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